YOUGOV
1 Intoduction
This report is one of 493,845 that I will make, and one of 104,070,413 that could be made.
I “toke” the 1.4 TB Linked-In data that was breached in 2020, and turned it into some insights to power my job HUNT.
The insights I could share in this report, that are also related to my goals, are:
- Industry base recruitment trend.
- Company base workforce timeline.
- Current/part workforce info:
- Basic info: Name, job title, status, social link. I could add geo-location for some that have the data, but it would look creepy.
- Their work period.
- Their experiences.
2 About me
Salutations; I’m Joseph, a self-taught data analyst, engineer, and scraper.
Despite life’s challenges, my goal remains a remote job, either full or part-time, and having friends to tackle the challenges of this changing world with.
To show my skills and dedication, I made this project that yielded this tailored report.
3 About the project
3.1 How this project comes to life?
You would know by now, from my email, that I am hunting for a job.
About a year ago, I scraped contact info from Google Map to get my first job. Later I scraped contact from Linked-In website… you can check how that went in here.
Recently, I finally got to learning SQL because of DuckDB, it is a software that allows you to process big data in your local machine by using storage space as RAM; Then I remembered about a leaked Linked-In data that I couldn’t process.
Thus my journey started to learn SQL, process the data, and make something out of it.
3.2 The process
The process was done in my local machine, and it was as followed.
3.2.1 Downloaded the leaked data
I downloaded the data from a torrent.
There was around 700 .gz file, each is around 280 Mb; 196 GB in total.
Each .gz file contain a 2 GB file; 1.4 TB in total.
Each file have multiple lines, and each one of them is a JSON; Not the file is a JSON, it just have multiple JSONs, one in each line.
3.2.2 Processing the weird data
I this phase I created a script that automatically open an archive, process the file, and save it as a Parquet file with compression level of 22.
I used Python, Pathlib, Polars, and a lot of patience.
The process toke around 20 minutes per file, in total it toke around three weeks (I had to shutdown my PC at night) The result was 700 parquet files, each is around 190 Mb; 133 GB in total.
3.2.3 Making relational database
The data in the datasets were nested, especially the “experience” field, it had the experience of a person and the company info; The problem is that the company info get repeated multiple tiles, across all datasets.
Making a relational database will solve this, and make the exploratory data analysis easier.
The code was split in two:
1. I used Polars to split each of the 700 datasets into mini relational databases.
2. I used DuckDB to merge all the mini relational databases and remove duplicates in some, mainly company and university information’s.
The result was a relational database that is 73 GB in size; From 1.4 TB to 73 GB.
All of this is using my PC, so servers were harmed, only my CPU fan and my ear.
3.2.4 Filter
I filtered out companies base on their industry, country, and whether I have the email of one of the higher ups.
4 General graphs
4.1 market research indestry’s yearly new recruit count
4.2 yougov’s workforce status over the years
5 Workforce sample
5.1 Abdullah Almusaed
Job title: Field manager
Associated: True
Socials: https://linkedin.com/in/abdullah-almusaed-367a0453
5.1.1 Abdullah Almusaed’s working period at yougov
5.1.2 Gantt plot of Abdullah Almusaed’s experience
5.2 Alan Aviles
Job title: Senior analyst - data products
Associated: True
Socials: https://linkedin.com/in/alan-aviles-25872412 | https://facebook.com/aviles | https://linkedin.com/in/alanaviles
5.2.1 Alan Aviles’s working period at yougov
5.2.2 Gantt plot of Alan Aviles’s experience
5.3 Amy Goldenberg
Job title: Business analyst
Associated: True
Socials: https://linkedin.com/in/amy-tebbi-14493210 | https://linkedin.com/in/amy-goldenberg-14493210
5.3.1 Amy Goldenberg’s working period at yougov
5.3.2 Gantt plot of Amy Goldenberg’s experience
5.4 Anar Memmedov
Job title: ****
Associated: True
Socials: https://linkedin.com/in/anar-memmedov-174441184
5.4.1 Anar Memmedov’s working period at yougov
5.4.2 Gantt plot of Anar Memmedov’s experience
5.5 Andrea Beck
Job title: Sales director
Associated: True
Socials: https://linkedin.com/in/andrea-beck-164902 | https://linkedin.com/in/beckandrea | https://facebook.com/jokiebird | https://quora.com/andrea-beck
5.5.1 Andrea Beck’s working period at yougov
5.5.2 Gantt plot of Andrea Beck’s experience
5.6 Andrew Greiner
Job title: Executive editor, north america
Associated: True
Socials: https://linkedin.com/in/andrew-greiner-5595594 | https://facebook.com/andrew.greiner
5.6.1 Andrew Greiner’s working period at yougov
5.6.2 Gantt plot of Andrew Greiner’s experience
5.7 Andy Wong
Job title: Delivery manager, it development asia pacific
Associated: True
Socials: https://linkedin.com/in/andy-wong-2a40222b
5.7.1 Andy Wong’s working period at yougov
5.7.2 Gantt plot of Andy Wong’s experience
5.8 Anna Murphy
Job title: Product lead, syndicated data products
Associated: True
Socials: https://twitter.com/annacmurphy | https://linkedin.com/in/anna-murphy-08a05817
5.8.1 Anna Murphy’s working period at yougov
5.8.2 Gantt plot of Anna Murphy’s experience
5.9 Ashley Grosse
Job title: Senior vice president
Associated: True
Socials: https://linkedin.com/in/ashley-grosse-48b3721 | https://facebook.com/ashleycgrosse
5.9.1 Ashley Grosse’s working period at yougov
5.9.2 Gantt plot of Ashley Grosse’s experience
5.10 Ashley Martin
Job title: Non executive director and chair of audit and risk committee
Associated: True
Socials: https://crunchbase.com/person/ashley-martin | https://linkedin.com/in/ashley-martin-a2226615 | https://linkedin.com/in/ashleygrahammartin
5.10.1 Ashley Martin’s working period at yougov
5.10.2 Gantt plot of Ashley Martin’s experience
5.11 Ba Tilly Heald
Job title: Group head of governance and company secretary
Associated: True
Socials: https://linkedin.com/in/tillyheald
5.11.1 Ba Tilly Heald’s working period at yougov
5.11.2 Gantt plot of Ba Tilly Heald’s experience
5.12 Borys Wojciechowski
Job title: Android developer
Associated: True
Socials: https://github.com/borwoj | https://linkedin.com/in/boryswojciechowski
5.12.1 Borys Wojciechowski’s working period at yougov
5.12.2 Gantt plot of Borys Wojciechowski’s experience
5.13 Christopher Melchiorre
Job title: Vice president
Associated: True
Socials: https://linkedin.com/in/christopher-melchiorre-531a1111
5.13.1 Christopher Melchiorre’s working period at yougov
5.13.2 Gantt plot of Christopher Melchiorre’s experience
5.14 Colin Marson
Job title: Chief executive officer apac
Associated: False
Socials: https://linkedin.com/in/colin-marson-a3b15b4
5.14.1 Colin Marson’s working period at yougov
5.14.2 Gantt plot of Colin Marson’s experience
5.15 Craig Eller
Job title: Contract creative
Associated: True
Socials: https://linkedin.com/in/craigeller | https://linkedin.com/in/craig-eller-aaa522a
5.15.1 Craig Eller’s working period at yougov
5.15.2 Gantt plot of Craig Eller’s experience
5.16 Dan Brilot
Job title: Media consulting director
Associated: False
Socials: https://linkedin.com/in/dan-brilot-73bb493 | https://twitter.com/brilotd | https://linkedin.com/in/danbrilot
5.16.1 Dan Brilot’s working period at yougov
5.16.2 Gantt plot of Dan Brilot’s experience
5.17 Delia Bailey
Job title: Senior vice president of data science
Associated: True
Socials: https://gravatar.com/deliabailey | https://github.com/tciqfban | https://klout.com/deliabailey | https://linkedin.com/in/delia-bailey-b298ab18 | https://twitter.com/deliabailey
5.17.1 Delia Bailey’s working period at yougov
5.17.2 Gantt plot of Delia Bailey’s experience
5.18 Doss Struse
Job title: Managing director
Associated: False
Socials: https://facebook.com/doss.struse1 | https://linkedin.com/in/doss-struse-45231 | https://linkedin.com/in/doss-struse-850992a
5.18.1 Doss Struse’s working period at yougov
5.18.2 Gantt plot of Doss Struse’s experience
5.19 Elizabeth Duarte
Job title: Doctor en medicina
Associated: True
Socials: https://linkedin.com/in/elizabeth-duarte-7394b9179
5.19.1 Elizabeth Duarte’s working period at yougov
5.19.2 Gantt plot of Elizabeth Duarte’s experience
5.20 Evan Williams
Job title: Director
Associated: True
Socials: https://linkedin.com/in/evaninsights | https://linkedin.com/in/evan-williams-1264495
5.20.1 Evan Williams’s working period at yougov
5.20.2 Gantt plot of Evan Williams’s experience
5.21 Fernando Gutierrez
Job title: Director of data integration
Associated: True
Socials: https://facebook.com/fernando.gh | https://linkedin.com/in/fernando-gutierrez-6058b33 | https://twitter.com/xbito | https://linkedin.com/in/fernandogh | https://youtube.com/user/xbitope | https://stackoverflow.com/users/76975 | https://angel.co/fernando-gutierrez | https://github.com/xbito
5.21.1 Fernando Gutierrez’s working period at yougov
5.21.2 Gantt plot of Fernando Gutierrez’s experience
5.22 Gavin Schneider
Job title: Cashier
Associated: True
Socials: https://linkedin.com/in/gavin-schneider-aa298a162
5.22.1 Gavin Schneider’s working period at yougov
5.22.2 Gantt plot of Gavin Schneider’s experience
5.23 Giles Haynes
Job title: Sales director anz
Associated: True
Socials: https://linkedin.com/in/gileshaynes | https://linkedin.com/in/giles-haynes-22763316
5.23.1 Giles Haynes’s working period at yougov
5.23.2 Gantt plot of Giles Haynes’s experience
5.24 Hasan Yesil
Job title: ****
Associated: True
Socials: https://linkedin.com/in/hasan-yesil-ab309b182
5.24.1 Hasan Yesil’s working period at yougov
5.24.2 Gantt plot of Hasan Yesil’s experience
5.25 Hossain Miah
Job title: Computerprogrammierer
Associated: True
Socials: https://linkedin.com/in/hossain-miah-6b70a3189
5.25.1 Hossain Miah’s working period at yougov
5.25.2 Gantt plot of Hossain Miah’s experience
5.26 Iris Yu
Job title: Business development manager
Associated: True
Socials: https://linkedin.com/in/iris-yu-57a373105
5.26.1 Iris Yu’s working period at yougov
5.26.2 Gantt plot of Iris Yu’s experience
5.27 J.J Helland
Job title: Partnerships + public relations
Associated: True
Socials: https://twitter.com/jjhelland | https://linkedin.com/in/j-j-helland-b039929 | https://linkedin.com/in/jjhelland
5.27.1 J.J Helland’s working period at yougov
5.27.2 Gantt plot of J.J Helland’s experience
5.28 James Rydberg
Job title: Product director
Associated: True
Socials: https://stackoverflow.com/users/2176262 | https://github.com/jamesrydberg | https://linkedin.com/in/james-rydberg-57474127 | https://linkedin.com/in/jamesrydberg
5.28.1 James Rydberg’s working period at yougov
5.28.2 Gantt plot of James Rydberg’s experience
5.29 Jillian Rowen
Job title: Analyst
Associated: True
Socials: https://linkedin.com/in/jillianrowen | https://linkedin.com/in/jillian-rowen-a7227010 | https://indeed.com/me/jillianrowen | https://facebook.com/1342876913
5.29.1 Jillian Rowen’s working period at yougov
5.29.2 Gantt plot of Jillian Rowen’s experience
5.30 John Blake
Job title: Consulting director
Associated: False
Socials: https://linkedin.com/in/johnblake3 | https://facebook.com/john.blake.16
5.30.1 John Blake’s working period at yougov
5.30.2 Gantt plot of John Blake’s experience
5.31 John Maiolino
Job title: Respondent
Associated: False
Socials: https://linkedin.com/in/john-maiolino | https://linkedin.com/in/john-maiolino-8213b083 | https://linkedin.com/in/johnmaiolino
5.31.1 John Maiolino’s working period at yougov
5.31.2 Gantt plot of John Maiolino’s experience
5.32 Josh Hawk
Job title: Yo
Associated: True
Socials: https://linkedin.com/in/josh-hawk-3a06bb125
5.32.1 Josh Hawk’s working period at yougov
5.32.2 Gantt plot of Josh Hawk’s experience
5.33 Ka Yo
Job title: Kam
Associated: True
Socials: https://linkedin.com/in/ka-yo-281b38127
5.33.1 Ka Yo’s working period at yougov
5.33.2 Gantt plot of Ka Yo’s experience
5.34 Karina Weitzer
Job title: Project manager
Associated: True
Socials: https://linkedin.com/in/karina-weitzer-759abb44
5.34.1 Karina Weitzer’s working period at yougov
5.34.2 Gantt plot of Karina Weitzer’s experience
5.35 Kenton Barello
Job title: Director, enterprise sales
Associated: True
Socials: https://linkedin.com/in/kenton-barello-1677674 | https://facebook.com/kenton.barello | https://linkedin.com/in/kentonbarello
5.35.1 Kenton Barello’s working period at yougov
5.35.2 Gantt plot of Kenton Barello’s experience
5.36 Kevin Sansone
Job title: Senior director
Associated: True
Socials: https://facebook.com/kevin.sansone.7 | https://linkedin.com/in/kevin-sansone-3418764
5.36.1 Kevin Sansone’s working period at yougov
5.36.2 Gantt plot of Kevin Sansone’s experience
5.37 Larry Shannon-Missal
Job title: Head of realtime research, us
Associated: True
Socials: https://linkedin.com/in/larry-shannon-missal-4512287 | https://linkedin.com/in/lsmissal | https://facebook.com/larryshannonmissal
5.37.1 Larry Shannon-Missal’s working period at yougov
5.37.2 Gantt plot of Larry Shannon-Missal’s experience
5.38 Leon Yer
Job title: Deputy head of panel, asia-pacific
Associated: True
Socials: https://linkedin.com/in/leon-yer-a382a645
5.38.1 Leon Yer’s working period at yougov
5.38.2 Gantt plot of Leon Yer’s experience
5.39 Lily Yan
Job title: Head of west coast accounts
Associated: True
Socials: https://linkedin.com/in/lilyyan32 | https://facebook.com/lillers.yan
5.39.1 Lily Yan’s working period at yougov
5.39.2 Gantt plot of Lily Yan’s experience
5.40 Liviu Popa
Job title: Panel executive
Associated: True
Socials: https://linkedin.com/in/lipopa | https://linkedin.com/in/liviu-popa-8a3932a8
5.40.1 Liviu Popa’s working period at yougov
5.40.2 Gantt plot of Liviu Popa’s experience
5.41 Marzena Gadomska
Job title: Research manager
Associated: False
Socials: https://linkedin.com/in/marzena-gadomska-9a6331b
5.41.1 Marzena Gadomska’s working period at yougov
5.41.2 Gantt plot of Marzena Gadomska’s experience
5.42 Matthew Britner
Job title: Director, key accounts
Associated: True
Socials: https://linkedin.com/in/mbritner | https://linkedin.com/in/matthew-britner-35405826
5.42.1 Matthew Britner’s working period at yougov
5.42.2 Gantt plot of Matthew Britner’s experience
5.43 Mervin Boey
Job title: Director
Associated: True
Socials: https://linkedin.com/in/mervin-boey-a853728
5.43.1 Mervin Boey’s working period at yougov
5.43.2 Gantt plot of Mervin Boey’s experience
5.44 Michael Nahoum
Job title: Strategic accounts director
Associated: True
Socials: https://linkedin.com/in/mnahoum
5.44.1 Michael Nahoum’s working period at yougov
5.44.2 Gantt plot of Michael Nahoum’s experience
5.45 Nate Becker
Job title: Survey analyst intern
Associated: False
Socials: https://youtube.com/user/nathanielbecker | https://angel.co/nate-becker | https://facebook.com/nathaniel.becker | https://twitter.com/oynate | https://meetup.com/members/75762602 | https://github.com/nathanielbecker | https://klout.com/user/id/75998249114153967 | https://linkedin.com/in/nathanielbecker
5.45.1 Nate Becker’s working period at yougov
5.45.2 Gantt plot of Nate Becker’s experience
5.46 Nathan Gelman
Job title: Senior marketing research analyst
Associated: True
Socials: https://facebook.com/nate.gelman.7 | https://linkedin.com/in/nathan-gelman-59b87556
5.46.1 Nathan Gelman’s working period at yougov
5.46.2 Gantt plot of Nathan Gelman’s experience
5.47 Pati Val
Job title: Diretor nã o executivo
Associated: True
Socials: https://linkedin.com/in/pati-val-a34511171
5.47.1 Pati Val’s working period at yougov
5.47.2 Gantt plot of Pati Val’s experience
5.48 Phil Rance
Job title: Managing director, uk
Associated: False
Socials: https://linkedin.com/in/philrance | https://twitter.com/philrance
5.48.1 Phil Rance’s working period at yougov
5.48.2 Gantt plot of Phil Rance’s experience
5.49 Pia Fruergaard
Job title: Nordic operations manager
Associated: True
Socials: https://linkedin.com/in/pia-fruergaard-350b9a4 | https://facebook.com/pia.fruergaard
5.49.1 Pia Fruergaard’s working period at yougov
5.49.2 Gantt plot of Pia Fruergaard’s experience
5.50 Rahul Singhal
Job title: Year student
Associated: True
Socials: https://linkedin.com/in/rahul-singhal-133a70189
5.50.1 Rahul Singhal’s working period at yougov
5.50.2 Gantt plot of Rahul Singhal’s experience
5.51 Rebecca Phillips
Job title: Senior director of analytics
Associated: True
Socials: https://linkedin.com/in/rebeccaaphillips
5.51.1 Rebecca Phillips’s working period at yougov
5.51.2 Gantt plot of Rebecca Phillips’s experience
5.52 Roger Payne
Job title: Key account director
Associated: True
Socials: https://facebook.com/7718101 | https://linkedin.com/in/roger-payne-0198ba4
5.52.1 Roger Payne’s working period at yougov
5.52.2 Gantt plot of Roger Payne’s experience
5.53 Ruth Mathias
Job title: Global head of sector marketing
Associated: True
Socials: https://linkedin.com/in/ruth-mathias-15784a26 | https://facebook.com/ruth.mathias.5 | https://linkedin.com/in/ruthmathias
5.53.1 Ruth Mathias’s working period at yougov
5.53.2 Gantt plot of Ruth Mathias’s experience
5.54 Ryan Gross
Job title: Head of east coast accounts, data products
Associated: True
Socials: https://linkedin.com/in/ryangross1
5.54.1 Ryan Gross’s working period at yougov
5.54.2 Gantt plot of Ryan Gross’s experience
5.55 Saqi Sheikh
Job title: Group human resources director
Associated: False
Socials: https://linkedin.com/in/saqi-sheikh-a45b2613
5.55.1 Saqi Sheikh’s working period at yougov
5.55.2 Gantt plot of Saqi Sheikh’s experience
5.56 Shohag Dey
Job title: Director
Associated: True
Socials: https://linkedin.com/in/shohagdey | https://linkedin.com/in/shohag-dey-55b0723
5.56.1 Shohag Dey’s working period at yougov
5.56.2 Gantt plot of Shohag Dey’s experience
5.57 Todd Soley
Job title: Director of busniess development management
Associated: False
Socials: https://linkedin.com/in/todd-soley-64b5775 | https://twitter.com/toddsoley | https://facebook.com/toddsoley
5.57.1 Todd Soley’s working period at yougov
5.57.2 Gantt plot of Todd Soley’s experience
5.58 Victor Gras
Job title: Head of account management us
Associated: True
Socials: https://linkedin.com/in/grasvictor | https://facebook.com/vgras
5.58.1 Victor Gras’s working period at yougov
5.58.2 Gantt plot of Victor Gras’s experience
5.59 Vũ Ken
Job title: Ä jj
Associated: True
Socials: https://linkedin.com/in/vũ-quốc-ken-9b8930189
5.59.1 Vũ Ken’s working period at yougov
5.59.2 Gantt plot of Vũ Ken’s experience
5.60 Yifei Liu
Job title: Marketing science manager
Associated: True
Socials: https://linkedin.com/in/yifei-liu-22b0a870 | https://linkedin.com/in/liuyifei